Carrickfergus
Merging Facts, Crafting Fallacies: Evaluating the Contradictory Nature of Aggregated Factual Claims in Long-Form Generations
Chiang, Cheng-Han, Lee, Hung-yi
Long-form generations from large language models (LLMs) contains a mix of factual and non-factual claims, making evaluating factuality difficult. To evaluate factual precision of long-form generations in a more fine-grained way, prior works propose to decompose long-form generations into multiple verifiable facts and verify those facts independently. The factuality of the generation is the proportion of verifiable facts among all the facts. Such methods assume that combining factual claims forms a factual paragraph. This paper shows that the assumption can be violated due to entity ambiguity. We show that LLMs can generate paragraphs that contain verifiable facts, but the facts are combined to form a non-factual paragraph due to entity ambiguity. We further reveal that existing factual precision metrics, including FActScore and citation recall, cannot properly evaluate the factuality of these non-factual paragraphs. To address this, we introduce an enhanced metric, D-FActScore, specifically designed for content with ambiguous entities. We evaluate the D-FActScores of people biographies generated with retrieval-augmented generation (RAG). We show that D-FActScore can better assess the factuality of paragraphs with entity ambiguity than FActScore. We also find that four widely used open-source LLMs tend to mix information of distinct entities to form non-factual paragraphs.
- Asia > Singapore (0.05)
- North America > United States > Michigan (0.04)
- Europe > United Kingdom > England (0.04)
- (23 more...)
- Leisure & Entertainment > Sports (1.00)
- Media (0.94)
- Government > Regional Government > North America Government > United States Government (0.67)
Agent-based models of social behaviour and communication in evacuations: A systematic review
Templeton, Anne, Xie, Hui, Gwynne, Steve, Hunt, Aoife, Thompson, Pete, Köster, Gerta
Most modern agent-based evacuation models involve interactions between evacuees. However, the assumed reasons for interactions and portrayal of them may be overly simple. Research from social psychology suggests that people interact and communicate with one another when evacuating and evacuee response is impacted by the way information is communicated. Thus, we conducted a systematic review of agent-based evacuation models to identify 1) how social interactions and communication approaches between agents are simulated, and 2) what key variables related to evacuation are addressed in these models. We searched Web of Science and ScienceDirect to identify articles that simulated information exchange between agents during evacuations, and social behaviour during evacuations. From the final 70 included articles, we categorised eight types of social interaction that increased in social complexity from collision avoidance to social influence based on strength of social connections with other agents. In the 17 models which simulated communication, we categorised four ways that agents communicate information: spatially through information trails or radii around agents, via social networks and via external communication. Finally, the variables either manipulated or measured in the models were categorised into the following groups: environmental condition, personal attributes of the agents, procedure, and source of information. We discuss promising directions for agent-based evacuation models to capture the effects of communication and group dynamics on evacuee behaviour. Moreover, we demonstrate how communication and group dynamics may impact the variables commonly used in agent-based evacuation models.
- North America > United States > New York > New York County > New York City (0.14)
- North America > United States > California > Los Angeles County > Beverly Hills (0.04)
- Europe > Switzerland (0.04)
- (16 more...)
- Overview (0.93)
- Research Report > New Finding (0.92)
Could NASA now stop a doomsday asteroid hitting Earth?
The idea of intentionally smashing a spacecraft into an asteroid may evoke memories of science fiction blockbusters such as Armageddon or Deep Impact. But it became a reality last week, as NASA's Double Asteroid Redirection Test (DART) spacecraft soared into a small asteroid called Dimorphos at 14,000mph, with the aim of tweaking its orbit. The space agency hopes the technology could one day be used to defend Earth against a'doomsday' asteroid or comet. But could we really avoid a future Armageddon if we detected a huge space rock headed our way? MailOnline's Executive Science and Technology Editor, Shivali Best, investigates.
- Europe > Ukraine (0.05)
- North America > United States > California > Santa Barbara County > Santa Barbara (0.05)
- Europe > United Kingdom > Northern Ireland > County Antrim > Carrickfergus (0.05)
- (3 more...)
- Government > Space Agency (1.00)
- Government > Regional Government > North America Government > United States Government (1.00)
- Asia > Japan > Honshū > Kantō > Tokyo Metropolis Prefecture > Tokyo (0.14)
- North America > Canada > Alberta (0.13)
- Asia > Japan > Honshū > Chūbu > Toyama Prefecture > Toyama (0.04)
- (41 more...)
- Media > Television (1.00)
- Media > Music (1.00)
- Media > Film (1.00)
- (9 more...)